Future Of Search

The future of search, and of searching online, could be drastically different, and better, from what we have now. If you only wanted to.

Photo credit: Eyeball

As we head further into the future, search engines, as we know them today, will increasingly play an important and critical role for society as they invisibly determine which world views, information and references we should look at and rely on when in need to make important decisions.

Consider for example the fact that Google defines itself now as a semantic search engine. In simple words, this means that Google attempts to interpret what you really want instead of just serving you with a list of web pages that contain specific keywords.

The risky consequence of this is that Google, and any other search engine taking the same approach, will increasingly try to guess what you want and what is right for you, without ever letting you have a direct hand into this decision-making process.

But for a company (Google) that is also the market leader in search advertising, establishing a high level of credibility when providing search results could become an uphill effort. When in fact commercial advertising objectives do not match up with search user expectations and requests for greater transparency, it becomes a natural possibility that, if alternatives emerge, users will start to test and consider them.

But what alternatives and options do we really have?

What can we do to move the quality of search to a whole new level of granularity, transparency and effectiveness while shredding off any attempts to influence or altogether manipulate search results for specific business interests?

In my view, the best solution(s) could likely be found exactly where we could less likely look.

Let me explain why.

1) What's Wrong with Google Search

To identify a new road and possible alternatives, I think it is necessary to put in good focus, what is wrong in the present search system, so that it becomes easier to identify solutions that address and resolve those specific limitations.

Here a short list of what I see as the main issues with the existing search system.

1. Quality of Search Results

Depending on the topic or industry you happen to be searching in, your mileage may vary, but in my personal experience the quality of Google search results cannot really be rated as ideal.

Often results are representative more of big brands and companies operating in the space, rather than being a true collection of the best resources on the topic.

Here some relevant comments and opinions:

"...higher quality sites are often further down in the search results because they're not as popular as the sites that are ranked higher".

Source: Is Google Dumbing Down Search Results? by Chris Crum, September 2013 - WebProNews

"...This is part of why I think I've developed a reflex, after searching Google, to skip over the first few results after the sponsored links and start looking near the middle of the page. W3Schools, Wikipedia, and a few others. And it's a great example of the central failure of the pagerank idea: if the strongest signal is popularity measured through linkage, the highest quality results will rarely be at or even near the top".

Source: Lee Philips comment to the article Why I'm Planning to Kill W3Schools, September 2013 -YCombinator

"Google ranks results by popularity (by how many sites link to each result). This isn't necessarily the same as quality.

Google individualizes search results and may connect you to information that fits with your past searches rather than providing a balanced view of a topic."

Source: Finding Quality on the Internet, September 2013 - Laurier Library

"...what is authority to the Google algo, is not what is authoritative to a human, and what its measure of quality is not human either".
By Graeme_p

"The search engine may or may not come up with the best site...
Google are delivering what will satisfy the majority of searchers and not the most accurate, practical or relevant information".
By EditorialGuy

"It really doesn't matter whether Google can measure quality or not, because it's only a very minor factor in their current ranking system. They had to de-emphasize it because it was preventing them from getting big brands and big organizations to the top of their search results."
By Aakk9999

Source: Authority and Quality: Google Definitions vs Common Sense , July 2013 - WebMasterWorld

See also: Declining Quality of Google Search Results

 

2. Ads Intrusion

The number of "paid for" ads inside Google search results, as well as the amount of other Google information blocks complementing the basic results has significantly increased, to dominate under certain circumstances, most of the search result page real estate.

Ads have also become somewhat less distinguishable from the "organic" search results, reducing immediate recognition and differentiation between "paid for" results and organic ones even more.

See also: Google Gets Lowest Score Ever In Customer Satisfaction Survey

 

3. Credibility

a) Automation

Google relies heavily on automation, and algorithms to index, organize and rank search results or it would not be possible for it to scale with the amount of content existing online.

At the same time, automation and algorithms are generally considered of a lower quality when compared to human analysis, especially when it comes to evaluating the quality and credibility of an information object. In this respect, Google may have already reached the peak of what can be done by solely automating the organization and ranking of search results.

And one of the reasons of why the Google superpower may be soon losing its mojo is because
people are increasingly much less interested in pure listings of relevant web pages created by an algorithm that tries to guess what's best, and much more in finding bundles of selected resources suggested to them by trusted references. Possibly human beings, they know (directly or not).

b) Conflict of Interest

Google is in control of what information you see when you search for something and how this information bits are ranked and organized. Wheter you like it or not it helps you build a world view and may influence it by selecting for you criteria by which to rank its results.

My question then is:

How can Google remain credible if,

a) It is totally secretive about it, even in the face of its own mistakes?

b) It is the world dominant search advertising platform, which pivots 100% around Google search engine, and which makes the majority of Google earnings?

That is: if you have a monopoly of the search market and control the search results, whether for the good or the bad, you can control the advertising market to your own benefit.

Then, while you could be saintest of the saints, how credible can you then be in such situation?

Link: The Google Panda Guide: Part 4 - The Future I Would Like To See

"The world's dominant search engine has been the focus of a European commission investigation since November 2010, after more than a dozen complainants across Europe accused the company of promoting its own services at their expense."

Source: Google Offers Further Search Result Concessions in EU Anti-trust Case by Foo Yun Chee, September 2013 - Reuters

See also:

• Authority and Quality: Google Definitions vs Common Sense
• Does Google Know What's Authoritative to a Human?
• Is Google Search Manipulation Hurting Consumers?
• Why does Google Search Suck so Bad Now?

 

4. Secretness

Google has always been very secretive about its search ranking algorithm to avoid unscrupulous marketers to exploit it to gain visibility in search results. While one can understand the logic of this approach, the results of undertaking it are also in front of everyone.

a) There are probably more people investing very significant time and resources to game Google than those who do not.

b) Google search results do not offer a qualitative search experience as they are often dominated by big brands and not by sites and pages that provide true, valuable information.

c) Google keeps investing large amount of resources to counter this gigantic and ever-increasing effort to spam and "game the system" with limited (in my opinion) results so far.

d) Due to the above listed items, and due to the vast damage done to quality content web sites, and the limited results in improving the quality of search results, public trust in Google's ability to truly distinguish high-value content from spam and web site scrapers is generally quite low.

"While this is presumably done to prevent people from gaming the system (or competitors from copying features), it makes it a lot harder to determine whether Google is unfairly penalizing websites..."

"As the Electronic Frontier Foundation points out in a blog post criticizing the move, Google's search algorithms are opaque by design, and so there is no way of knowing what kind of criteria they will be using to decide which sites to penalize and which to leave untouched."

Source: Should We Trust Google When It Comes to Piracy and Search? by Mathew Ingram, August 2012 - Gigaom

When so much of our life depends on having instant access to the right information, don't you think it would be very risky to depend on a centralized and secret system, driven exclusively by financial gains, to continuously influence how information is organized, ranked and classified?

In my view everything should be transparent. No company or brand should be able to game the system without being vulnerable to everyone seeing it.

 

5. Controls, Features

Easy access to valuable functions like search according to type of sources, timing, visual search, preview of search pages, have easily come and gone from Google in favor of providing a search solution that adapts itself to the minimum common denominator, while silently imposing a de facto rigidity in restricting user search abilities.

Shouldn't Google offer me more search options rather than deciding for me the ones that I really need?

 

6. Fast Food Answers

Duration: 2' 06'' - What's the Role of Curation in Search - Oliver Starr talkes about the impact of curated content on how people are now finding information on the Internet.

One thing is to have a full meal, one other is to fill yourself with edible stuff.

Google's background and strength is in providing you with a ranked list of web pages that contain the set of keywords you have specified. But between a set of keyword-matched web pages ranked according to secret rules, and a well-reviewed round-up of the best resources on a specific topic, there is quite some distance.

One does not just search for finding the closest Pizza place.

Search is increasingly a vehicle through which find out, discover and learn about most things that one is not familiar with. A new topic or subject in which one wants to become more competent.

And when this is the situation, we definitely need something more than a list of popular web sites ranked by a secret algo.

7. Personalization

If Google wants to customize search results in order for me to have a better user experience, why doesn't it allow me also to tweak and tune how those results are calculated and also how they are presented to me?

Google has demonstrated over time, with multiple public tests and versions of Google Search that integrated some of these functionalities, that it can indeed develop them effectively. Why then instead of removing altogether many such functionalities, doesn't it let the user (when the user wants to) to decide how results should be ranked, formatted and presented?

If Google's ultimate goal is to help me find easily what I am looking for, why not let me (if I want to) decide which are the variables and criteria I want to utilize to rank results instead of imposing a secret set of them on me? Why doesn't it let me choose how to view, display, browse, scan or navigate search results?

Google rigidness in providing search results exclusively in a top-down linear, page by page, text-based list offers little or no opportunity to support different modes to search and declares that there must be one solution fitting everyone.

All of this is in stark contrast with Google search personalization strategy, where search results are customized for each search user depending on its past history, preferences and network of contacts.

See also: Escape Your Search Engine Filter Bubble

 

8. Privacy

When you search on Google, or other major search engines, are you actually providing personal private data to commercial companies and possibly to intelligence agencies of different countries or are you just conducting a truly "private" search, where no data is tracked, stored and associated to your name, IP and hardware setup?

See also: Don't Track Us

 

2) The Profile of a New Search Tool

Given all the above issues, what could be the alternative search route available to us?

How could we escape the limitations imposed by the Google search model?

Proposal:

What would happen if it was me and you, individually, the ones who selected the criteria, ranking algorithms and penalization approaches to use to determine our search results?

My assumptions:

a) To make search results more useful, while becoming more trusted and much less vulnerable to being reverse-engineered and gamed by unscrupulous marketers, I don't think there is a need to make your search engine and your ranking algorithms secret.

b) Secrecy promotes and breeds black markets, underground work and a well-defined objective for everyone: uncover the secret. Reverse-engineer it. Game it.

c) Following the patterns we have seen at work elsewhere: from centralized to distributed; from top-down expert secrecy to crowdsourced, open-sourced and distributed co-operative participation.

Here is how I see my ideal future search engine:

1. Open-source
2. Distributed - P2P
3. User-controlled

Imagine if:

a) Search was similar to a free public service.

b) Users could see search ranking and filtering factors and, if they wanted, they could change them according to their own specific needs and preferences.

c) "Trusted search curators" for specific vertical information niches started to become themselves the new relevant results. The idea is that they could provide the needed "trust" and transparency to search results by co-creating curated collections on the topics in which they have already demonstrated a high level of competence.

d) An ecosystem of open-source public search algorithms, filters, aggregators and curated collections of sites and resources on specific topics emerged.

e) Content indexing became a distributed activity in the hands of we the users. With this approach, individual users contribute to index and add information into a shared database aggregating each user personal index.

In this fashion, users not ONLY would have greater control of what is actually indexed, but they would actually be creating a real search commons index - a collaborative effort by all users that is available to everyone. (An example of a distributed search engine - where peers collaborate to construct their search database - is the YaCy project. More info on this Wikipedia page).

In simple terms:

Turn the search ranking mechanism upside down by giving back control to who is searching and in need of taking decisions based on that information.

Achieve this by allowing the user to see at all times, what is under the hood and to have the option to modify it, rather than achieving this by personalizing his results univocally or by differentiating them from those of others based on history, preferences or the social graph.

 

3) Key Benefits

An ungameable system.

If everyone could individually select and rank results according either to their preferences, or by utilizing user-defined filtering pre-sets, ranking plugins done by experts and niche curators, it becomes much more difficult for anyone to game search engine results, as now there would be an infinite number of different ranking systems at work.

But unlike what Google does with personalization, the rules by which results are ranked are not secretely set by Google, but it is me and you who decide how we want to slice and dice them.

For those who wouldn't want to bother with tweaking and setting up their individual search preferences, they could be offered to select among alternative ranking algorithms (e.g.: Pre-Panda, 1998-style, etc.), or through open-source or paid-for algo pre-sets designed by users, groups or even other search engines themselves.

Commercial search engines like Google could even license their own algo as a plugin to use by paying a monthly subscription fee. Better yet, they could offer different flavours of it tailored to different audiences, applications and with different levels of customization possible.

Finally search users could also select to fallback on the preferences set by their close network of friends (on Facebook, Twitter, etc.) or even by Google (or other search engines) itself if they wished so

If you had no "online" friends and did not want to set preferences, paradoxically you could be given an alphabetical, or chronologically indexed set of results, and then you could move on to refine and distill what you need out of it, by applying on the fly, your own criteria.

If search was a distributed mechanism, there would be no central server, and thus there would be possibility to censor or block specific content.

Reliability would be significantly increased as single points of failure have been eliminated and the search index is stored redundantly across all users in the network that opt-in to participate.

It becomes possible to index and crawl contents, databases and web sites previously not accessible.

It is finally possible to achieve a high degree of privacy and to leverage the power of crowdsourced ranking.

 

4) What You Can Do About It

If you like the future of search that I am painting, you should not be sitting there waiting for it to happen. It won't.

This future can only take place if you stop what you have been doing until now and you start doing something different.

Here a few simple suggestions:

a) Vote with your click

The first and foremost thing you can do to change the status quo, when it comes to online search, is to vote with your clicks. If Google does not offer what is your "ideal" situation, then do not support its growth and power by increasing your use of it.

b) Use alternative search tools

Start using an alternative search engine, among the ones that I suggest later in this article. Stop using Google or at least start to use it in combination with other alternatives.

c) Curate collections of quality vetted resources

If you are a subject matter expert or just passionate about a specific topic take into serious consideration the idea of starting to curate resource collections that can offer a one-stop-shop for those looking for insight into that topic.

Examples:

• Elearning Examples
• Marshall McLuhan Speaks

d) Speak up, share what you find

Speak up, share, let others know. If you find better results by using one of the alternative solutions available, or if you discover something new or valuable that can help other people search in new, effective ways, share your information via social media so that others can pick up from it and move forward.

 

5) Resources & Tools / New Search Tools

New search engines that already integrate many of the ideas and recommendations I am suggesting:

• YaCy
  A free distributed search engine, built on principles of peer-to-peer (P2P) networks.

  "YaCy runs on each user's own computer, and each YaCy user is part of a large search network. Each user's computer creates its own, individual search indexes and rankings, so no central entity has control and results better reflect what the user is looking for over time. YaCy makes it easy to create a customized search portal, and it protects users' privacy by encrypting search terms before they leave the user's computer."

  Source: YaCy: It's About Freedom, Not Beating Google by Katherine Noyes, December 2011 - PCWorld

  See also:

  Wikipedia definition

  YaCy Project explained





• Faroo
  

  "FAROO is a universal web search engine based on peer-to-peer technology. It uses a distributed crawler that stores search data on users' computers instead of a central server. Whenever a user visits a website, it is automatically indexed and distributed to the network. Ranking is done by comparing usage statistics of users, such as web pages visited, amount of time spent on each page, and whether the pages were bookmarked or printed."

  Source: Wikipedia





• zResearch
  Curated, collaborative next-generation search engine, allowing great customization and filtering of results, clustering and auto-categorization, some truly stunning alternative data visualization options, and the ability to collaboratively curate and organize search results into multi-level collections.

  
  

• Blekko
  

  "Blekko is a search engine with the stated goal of providing better search results than those offered by Google Search, with results gathered from a set of 3 billion trusted webpages and excluded from such sites as content farms."

  Source: Wikipedia




• Gooru
  Search curated collections of high-quality web resources.




• Glean
  A search engine for the best video lessons in education.




• DuckDuckGo
  Search the internet without being tracked.




• Seeks
  Crowdsourced search results ranking




• Dooble
  An open source web browser that complements YaCy Search Engine as an indexing tool.

See also: User-Curated Search Engine

 

Conclusions

Search as we know it today, could be a million times better, if we only decided that we wanted it to be so.

While we associate searching the Internet with Google, the future of search could be quite different from what you have been used to until now.

In my view in fact, this approach to search, will not remain the major, most common and most effective search solution available in the digital realm.

What is really best for us?

A centralized, secret and proprietary search engine driven by Wall Street or a distributed, fully transparent and open-sourced one that placed each and every user / searcher in the driver seat?

Put the choice of how to rank Internet search results results in the hand of the searchers, not in the hands of those who control both the search and advertising marketplace.

Let users index, refine, develop and improve search engine ranking algorithms by applying the filters and metrics that serve THEM best, and not only the Google stock.

Move from listing titles-URLs-descriptions to curated search results, in which "trusted search curators" will provide bundles of high-quality results, selected and organized together in new emerging formats.

If Google and the other major search engines are not willing to be transparent about how they organize, filter and rank information, how can you trust that the answers you are given do really provide you with the best option possible?

Access to information should not be based on some "social" secret recipe of what is good and what is bad, - that is taken care of by religions of this world - as there is no objective metric that can measure the different needs and information requirements of each human being.

Unless I can check it.

But unless you decide that Google should not be your only key reference for finding any kind of information that exist out there, then nothing is going to change.

Reference:

• The Google Panda Guide: Part 4 - The Future I Would Like To See
• The Evil Side of Google

Duration: 5':00 - Gilad Lotan from BetaWorks explains to Lutz Finger (Author of "Ask Measure Learn") why powerful algorithms can be as well a danger to society.

Playlist - Content Curation and The Future of Search (8 clips)

 

Originally written and curated by Robin Good and first published on MasterNewMedia on Tuesday March 11th 2014 as Future of Search.

Photo credits:
What's Wrong with Google Search - Senior thinking by Shutterstock
The Profile of a New Search Tool - Identification sign by Shutterstock
Key Benefits - Check mark by Shutterstock
What You Can Do About It - Young man thinking by Shutterstock
Resources & Tools / New Search Tools - Zoom icons by Shutterstock

Robin Good -